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© System for combining multiple-format multiple-source video signals. 



© The system generates a composite image on a 
pixel-by-pixel basis by selectively mapping source 
pixels provided by video sources of different video 
formats onto destination pixels in the composite im- 
age. The mapping is accomplished by encoding the 
source pixels into digital words, each word having a 
colour information part and a video format identifier, 
used to decode the colour information part upon 
processing to organize the composite image. Com- 



bining video signals with computer-generated graph- 
ics data is achieved under control of the graphics 
data being interpreted either as video format iden- 
tifier or as colour. If the colour of the destination 
pixel is determined by one of the video sources, the 
graphics data is used to select the appropriate de- 
coding. The system permits a modular architecture 
and simple merging with a PC. 
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FIELD OF THE INVENTION 

The invention relates to a system for generat- 
ing a composite image by selectively combining a 
plurality of simultaneously available video signals 
with one another on a pixel-by-pixel basis. 

BACKGROUND ART 

ABltw S -£ t0m ' S kn ° Wn fr ° m US Pate "t 
4.876,600. c.iIhi-5 prior art reference discloses an 

apparatus^ produce a composite image from in- 
formation supplied by a plurality of independently 
and mutually asynchronously operating sources 
such as TV cameras, mass storage for digital im- 
age data and raster graphic generators. The sour- 
ces provide the information to a plurality of dual- 
ported buffers. The data stored in the buffers are 
read in a synchronized manner. A pre-programmed 
mask storage selects on a pixel-by-pixel basis 
which one of the buffers is to supply the informa- 
tion for a given pixel of the composite image. The 
mask storage furnishes control data words that 
each comprise a number of bits equal to that of the 
number of sources to be handled. That is. the data 
widtti which may be read in parallel corresponds 
exactly to the number of sources. One bit is set per 
data word to achieve this selectivity feature 

The prior art reference does not address the 

fn^l m w 0f ^ Si9na,S w mutua "* differently 
formatted. Generally, video signals occur in an 
analog representation, such as certain TV signals 
or in a digital representation such as computer- 
generated graphics. In addition, each such repre- 
sentation may have its information contents be 
encoded in a variety of ways, dependent on the 
signals intended use (display, editing, storage 
transmission, etc.). Following video signal types are* 
mentioned here by way of example: compressed 
still images (JPEG), uncompressed still images 
compressed moving images (MPEG), uncompres- 
sed moving images (PAL. NTSC. SECAM D2MAC 
HDMAC, YUV. Y/C. RGB, RS170A. SwKE 
patterns, etc. The video signals and the graphics 

OLTV?** t0 b6 re,ormatted f a common 
format before they can be simultaneously dis- 
played on a same display, e.g.. a PC's monitor 



tt is an object of the invention to provide a 
system for combining a plurality of video signals 
with one another, wherein real-time processing is 
feasible for practically an unlimited number of si- 

s multaneously available video signals from different 
sources, irrespective of their formats. It is another 
object of the invention to support a modular ap- 
proach with regard to the organization of operations 

w t0 56 P erf ormed and regarding system architecture. 

SUMMARY OF THE INVENTION 



OBJECT OF THE INVENTION 

The format issue briefly touched upon above 
may serve to indicate that selectively combining on 
a real-time basis a plurality of differently formatted 
simultaneously available and mutually asynchro-' 
-noir ^ vido o u iy uulj w i th . ono anothu i and/er-wtth- 



_ - «"iwuiui uiiu/UI WHO 

-^mculef ^enerated graphics p i™* to a nnn . 
tnvial handling of data. — 
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To achieve these goals, the invention provides 
a system for creating a composite image by selec- 
ts tively combining a plurality of simultaneously avail- 
able video signals with one another on a pixel-by- 
Pixel basis. The system comprises input means 
operative to receive the plurality of video signals 
each respective one of the video signals having a 

ZSS??* 0 ^ 3nd enC0di "9 means oon- 
nn~^f 1 l npUt meanS f ° r encodin 9 respective 
ones of the video signals into data items provided 
at an encoder output Each particular one of the 
data items corresponds to a particular source pixel 
*s and functionally has a colour identifier to ideXa 
colour of the particular source pixel and a format 
identifier to identify the respective video format 
The system further includes multiplexing means 
having a multiplexing input coupled to the encoder 
30 output for transferring a selected one of the data 
items to a multiplexing output, and memory means 
for storage of the selected data Hem. The memory 
means has a memory input coupled to the mul- 
tiplexmg output and has a memory output The 
as system also includes decoding means coupled to 
the memory output to receive the selected data 

U^J° deCOde tne colour id entifier of the 
selected data item for determining a colour of a 
specific one of destination pixels of the composite 

T 9 J! T?" COntro1 of the f0 ™at identifier of the 
selected data item. 

In the invention, source pixels are extracted 
from the video signals. The source pixels are re- 
presented as data items of a specific kind. Each 
45 data rtem »» colour information and format in- 
formation. The colour of a specific destination pixel 
of the composite image is determined by selectino 
one of the data items and decoding the selected 
data item s colour identifier under control of the 
so selected data item's format identifier. Prior to de- 
coding, the data items are organized in a memory 
means. 7 

The system of the invention permits a modular 
^"tecture. Using the common, predetermined re- 

ff"d_jy^^ 

tvne anri fnrmat *t • _i_ _ . ~ ,v 



type and format oftte video sig^teaTtiTmZ - 
means from the organizational processing in the 
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memory means. Further, any system supported 
internal format for encoding the data items can be 
adopted as long as the functionalities of colour key 
and format key are unambiguously and retrievably 
defined. The internal format can be adapted to, 
e.g., the memory organization and processing ca- 
pabilities of memory management, or to a suitable 
and predetermined window-based data handling in- 
struction set, or to another kind of preprocessing 
executed on the uniformly formatted data items. 
The use of the internal format also renders the 
system versatile regarding the format of the com- 
posite image. The composite image format is to be 
adapted to the destination device that displays, 
stores or processes the eventually generated com- 
posite image. Changing the destination device's 
format may occur, for example, when a monitor of 
one type, used for display of the composite image, 
is replaced by a monitor of another type. The 
replacement may need adaption to, e.g., the phys- 
ical characteristics of the new monitor, due to the 
particular phosphor compounds used in the moni- 
tor's screen ("gamma-correction"). Changing the 
format of the composite image is achieved through 
the decoding means, again without interfering with 
the organizational processing in the memory 
means. 

The system is readily modified to handle a 
larger number of simultaneously available video 
signals by appropriately extending the encoding 
means and the decoding means and the capability 
of the multiplexing means. In order to enhance the 
system's modularity, the multiplexing means pref- 
erably includes a bus means for under control of a 
selection protocol transferring the selected one of 
the data items to the decoding means. 

The format identifier may functionally be the 
same as a video source identifier when each in- 
dividual video source supplies video signals of only 
a single video format. Also, the number of bits 
used to encode the colour information per source 
pixel may depend on the specific type of video 
signal to be encoded. This may stem from the fact 
that a video signal of a particular type needs fewer 
bits per source pixel to represent the pixel's avail- 
able colour information than a video signal of an- 
other type. By providing adequate memory man- 
agement means to handle variable word lengths, an 
efficient use is made of the available memory 
space. 

Preferably, the system of the invention is oper- 
ative to selectively combine the video signals with 
computer-generated graphics data on a pixel-by- 
pixel basis. To this end, the system comprises 
c o ntro l means for specify i ng wh e th e r th o plural i ty 
of video signals or the computer-generated oxaph- 
ics data is to determine the colour of the specific 
destination pixel, and further multiplexing means 



coupled between the memory means and the de- 
coding means for determining the colour of the 
specific destination pixel under control of the con- 
trol means. The further multiplexing means either 
5 transfers the colour identifier supplied by the mem- 
ory means to the decoding means and transfers 
the graphics data to the decoding means for de- 
coding the colour identifier under control of the 
graphics data, the latter functioning as the format 

10 identifier, or transferring the graphics data to the 
decoding means for being decoded. 

::;The memory means of the system in the in- 
vention comprises a colour identifier section to 
store the colour identifier of the selected data item, 

75 and a format identifier section to store the format 
identifier of the selected data item. If the system 
includes a graphics data generator, such as a PC, 
preferably at least the format identifier section or 
the control means is implemented as a functional 

20 part of the graphics generator. In the system of the 
invention, the memory means has a source iden- 
tifier section to store a source identifier for control 
of the first-mentioned multiplexing means. The 
source identifier section then also preferably is 

25 implemented as a functional part of the graphics 
generator. 

Accordingly, the invention furnishes a system 
that permits combining video signals of various 
formats with computer generated graphics, prefer- 
30 ably in window-based representation. 

BRIEF DESCRIPTION OF THE DRAWING 

The invention is explained in more detail by 
35 way of example and with reference to the accom- 
panying drawing, wherein: 

Figures 1 and 2 disclose a first example of a 
system in the invention; 

Figures 3 and 4 illustrate a second example of a 
40 system in the invention; and 

Figures 5 and 6 show a third example of a 
system in the invention. 

Throughout the Figures, same reference nu- 
merals are used to indicate similar or correspond- 
45 ing features. 

DETAILED DESCRIPTION OF THE EMBODI- 
MENTS 

so General 

Figures 1-6 illustrate examples of a system 
according to the invention. The system creates a 
composite image out of a plurality of simultaneous- 

l y ava il ab le v ide o s ig na l s 11 , 1 2, .... I n. Video s i gna l s 

11 -In may have mutually different vi deo fo rmats. 

The composite image is a pattern of destination 
pixels that each have a particular colour or grey 
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level. The pattern is determined by selection of the 
video signals II -In on a pixel-by-pixel basis. For 
each of the destination pixels one of the video 
signals 11 -In is selected to supply a source pixel 
whose colour or grey level information then is to be 
mapped onto the destination pixel. 

The video signals may be originated in, e.g., 
TV receivers, VCRs, video cameras, video disks, 
etc. The video signals may include the following 
types and formats: compressed still images 
(JPEG), uncompressed still images, compressed 
moving imagesii\(MPEG), uncompressed moving 
images (PAL;_NTSC, SECAM, D2MAC, HDMAC, 
YUV. Y/C, RGB, RS170A, RS330). The signals may 
also be supplied by a video pattern generator or 
synthesizer. 

First example 

Figure 1 shows a first example of a system 100 
in the invention. System 100 comprises a multiple- 
source-multiple-format signal processing device 
(MSMF) 10 for generating a composite image from 
a plurality of simultaneously available video signals 
11 -In of one or more video formats. The composite 
image is transferred to an image destination device 
20, such as a memory or a monitor. Operation of 
MSMF 10 is controlled by a data processing ap- 
paratus 30. Apparatus 30 may comprise a PC 40 
provided with a user-interface 50, such as a key- 
board or a mouse, and a display 60 to enable 
interactive control of the image compilation. PC 40 
may comprise a graphics unit 70 having, e.g., a 
VGA and a RAM to store graphics data. 

Figure 2 shows MSMF 10 in more detail. 
MSMF 10 comprises the following functionalities: 
an encoding section 102, a display memory 104, a 
decoding section 106 and interface means 108 to 
PC 40. As explained below, encoding section 102 
encodes incoming video signals 11 -In into source 
pixels. The source pixels are supplied in one or 
more digital representations supported by the inter- 
nal processing of MSMF 10. The source pixels 
comprise colour and video format information. Dis- 
play memory 104 selectively stores the source 
pixels in a pattern corresponding to the pattern of 
destination pixels of the composite image. Decod- 
ing section 106 decodes the source pixel's colour 
information under control of the source pixel's vid- 
eo format information. The decoded colour informa- 
tion is presented in a common digital format for 
conversion into an analog signal adapted to monitor 
20. 

Encoding section 102 comprises a plurality of 
inputs 110, 11Z .... 114 to receive v ideo signals 11- 



of video signals 11 -In into respective sequences of 
digital words of one or more internal representa- 
tions, adapted to the data processing capability of 
MSMF 10. A multiplexing means 122 selectively 
5 transfers the words supplied by encoders 116-120 
to display memory 104. 

Encoders 116-120 each include a memory (not 
shown) of the DRAM-, VRAM- or TVRAM-type to 
store the encoded signals. In addition, these 

10 memories take care of the difference in the rates of 
pixel information supplied to the encoder's output 
and pixel information derived from the video signal 
received. The video information typically includes 
an analog signal sequence based on a given scan 

15 rate and provided with identifiers (sync pulses) to 
indicate individual video lines that make up a com- 
plete video image (frame) and to indicate individual 
frames. The monitor, onto which the video signals 
are to be displayed simultaneously, may have a 

20 different scan rate. Scan rate conversion then is to 
be accomplished to adapt the supply of video 
images to the monitor's operation. In addition, time 
base correction is required to remove time-skewing 
between the monitor's lines and frames on the one 

25 hand and the start of the video line and of the 
video frame on the other hand. 

The memories permit digital words from dif- 
ferent ones among encoders 116-120 to be read 
out on the same time basis. The memory may be a 

30 DRAM, but preferably is a VRAM (video RAM) or 
TVRAM (triple-ported VRAM). As known, a VRAM 
is a random-access-memory that has a bidirec- 
tional random access data port and a bidirectional 
sequential access data port. Multiple-ported RAMs 

35 such as VRAMs and TVRAMs are faster than or- 
dinary DRAMs and are very useful in video applica- 
tions due to high-speed capabilities and due to the 
fact that video data usually are sequentially or- 
ganized in pixels and lines. The VRAM is written 

40 according to an address pattern that corresponds 
to the scan pattern of the video image to be stored. 
That is, a sequence of fines of consecutive pixels 
of the image to be stored is mapped onto a group 
of memory cells of the VRAM that have logically 

45 adjacent memory addresses. The VRAM is read via 
a shift register that periodically is reloaded with the 
contents of such a group of cells to furnish the 
line's pixels consecutively. Reading and writing of 
the VRAM can be done at mutually different time 

so rates. A TVRAM further is adapted to be written 
and read at the same moment, it being provided 
with two (or more) shift registers to retrieve and 
enter pixel data in line fashion. 

Each word provided by encoders 116-120 re- 
-5§ presents a source pixel derived from the asso- 



ln, each having its individual video format. The 



received video signals 11 -In are respectively sup- 
plied to encoders 116, 118, .... 120 to encode each 



ciated video signal and includes several bi ts, a first 
part whereof contains encoded colour or grey level 
information for that source pixel, and a second part 
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whereof specifies the video format of the video 
signal to serve as a key for eventually decoding the 
first part later on. Conversion of an analog video 
signal such as a TV signal into a sequence of 
digital pixels is known in the art. Note, however, 
that in the invention the pixel's digital colour in- 
formation is combined with an identifier for the 
associated video format, and the combination is 
provided in a digital representation supported by 
MSMF 10. 

Display memory 104 receives the digital words 
from multiplexing means 122 on a pixel-by-pixel 
basis and functionally comprises following sections: 
a source information section 124, a colour informa- 
tion section 126 and a format indicating section 
128. Source information section 124 specifies in 
advance for each destination pixel of monitor 20 
which one of video signals 11 -In is to be selected to 
produce the source pixel for mapping onto the 
destination pixel. Section 124 controls the operation 
of multiplexing means 122. Colour information sec- 
tion 126 stores for each destination pixel the first 
part of the digital word received from multiplexing 
means 122, i.e., the colour information of the se- 
lected source pixel. Format indicating section 128 
stores for each destination pixel the second part of 
the digital word received from multiplexing means 
122, i.e., the video format indication for the se- 
lected source pixel. 

Display memory 104 may be implemented in 
various ways and may include a DRAM (dynamic 
RAM), a VRAM or a TVRAM. Such RAMs may be 
physically organized in several banks to increase 
storage capacity and access speed. Note that the 
speed per bank can be reduced when bank-switch- 
ing techniques are applied, such as storing data for 
subsequent destination pixels in logically contig- 
uous banks. Colour information section 126 and 
format indicating section 128 are preferably imple- 
mented with a one or more TVRAMs in view of the 
required fast-in-fast-out behaviour (high update 
rate). Source information section 124 preferably is 
implemented with one or more ordinary VRAMs or 
DRAMs, since updating of section 124 is typically 
done at a substantially tower rate than the updating 
of sections 126 and 128. 

Encoders 116-120 may be adapted to encode 
video signals received at inputs 110-114 in internal 
representations that are mutually different That is, 
encoders 116-120 may be made programmable to 
handle words of various widths. Trie number of bits 
used to encode the colour information per source 
pixel may be variable depending on the type of 
video signal to be encoded. This may stem from 
th e fact that a vid o o s i gn al of a p art i cu l ar type- 
needs fewer bits per source pixel to represent the 
pixel's available colour information than a video 
signal of another type. By providing section 126 in 



addition with adequate memory management 
means (not shown) to handle variable word lengths, 
an efficient use is made of the available memory 
space. 

5 Decoding section 106 receives the colour in- 

formation parts of the source pixels stored in sec- 
tion 126 and handles the conversion of the digital 
colour information into the analog format of monitor 
20. Decoding section 106 comprises a plurality of 
w decoders 1J0, 132, .... 134. Each different one of 
decoders^ 130-134 operates on the digitally en- 
coded;: cdlcur information parts of a different video 
format. "Therefore, when section 126 produces a 
colour information part of a source pixel, section 

75 128 supplies the associated format information part 
of that source pixel to select the associated one of 
decoders 130-134. 

For instance, format indicating section 128 may 
specify the colour information bits received by de- 

20 coding section 106 from section 126 to be an 
encoded representation of a pixel created from an 
analog signal in YUV format at one of inputs 110- 
114. The appropriate one of decoders 130-134 is 
selected to operate on that representation. The 

25 operation involves interpreting the colour informa- 
tion bits as encoded YUV information and convert- 
ing this information into a decoder output signal, 
whose format is adapted to the video format of 
monitor 20, e.g. analog RGB. The conversion re- 

30 quires a dedicated YUV/RGB converter that per- 
forms the true-colour space conversion to produce 
the RGB from the YUV. The decoder output signal 
is transferred by multiplexing means 136 to monitor 
20 under control of format indicating section 128. 

35 In another example, format indicating section 

128 may specify the colour information bits re- 
ceived by decoding section 106 from section 126 
to be an encoded representation of a pixel created 
from an analog signal in an RGB format at one of 

40 inputs 11Q-114. The appropriate one of decoders 
130-134 is selected to operate on that representa- 
tion. The operation involves interpreting the colour 
information bits as encoded RGB information and 
converting this information into an RGB decoder 

45 output signal, whose format is adapted to the video 
format of monitor 20. The RGB video signal at one 
of inputs 110-114 may need adaption to the phys- 
ical characteristics of monitor 20, due to the par- 
ticular phosphor compounds used in the monitor's 

so screen. This may be taken care of by a dedicated 
RGB/RGB decoder. This kind of operation is 
termed "gamma-correction*'. 

In still another example, format indicating sec- 
tion 128 may specify the colour information bits 

ss received b y decodii i g-sectiorr106-fr om s ec tl o rr126~ 

to be an encoded representation for the colour of a- 

destination pixel created from a signal at one of 
inputs 110-114 that does not represent a colour or 
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a grey level. This may be the case for data that are 
not directly related to visual perception. Examples 
are spatial temperature distribution, spatial humidity 
distribution, topologic surface characteristics or oth- 
er parametrized features, e.g., as obtained by re- 
mote sensing techniques from satellites, that are to 
be visualized. These data may be rendered visually 
perceptible by creating a plurality of parameter 
ranges for the feature to be visualized and display- 
ing the feature's spatial distribution by means of a 
spatial colour distnbtjjgoo;. a particular colour cor* 
responding to a pari«&Jar parameter range. This 
display technique employs pseudo-colours. In such 
pseudo-colour application, an appropriate one of 
decoders 130-134 is selected by section 128. The 
selected decoder is a pseudo-colour decoder and 
interprets the colour information bits from section 
126 as encoded non-colour information that is to be 
converted into a colour output signal, whose format 
is adapted to the video format of monitor 20. This 
interpretation and conversion may be accomplished 
by the decoder through a colour-look-up-table 
(CLUT). 

Preferably, the output signals supplied by de- 
coders 130-134 are digital signals, since multiplex- 
ing digital signals is simpler than multiplexing ana- 
log signals. A digital-analog converter (DAC) 138 
then is provided between an output of multiplexing 
means 136 and monitor 20. 

In the example shown in Figure 2, encoding 
section 102, decoding section 106 and display 
memory 104 are controlled by PC 40 via an inter- 
face means 108. Interface means 108 comprises a 
bus 140 for read/write data transport and control 
data transport. Bus 140 is connected to an encod- 
ing control unit 142, a still-picture control unit 144, 
a memory control unit 146 and a decoding control 
unit 148. 

Encoding control unit 142 serves to control the 
operation of encoders 116-120. For instance, a 
modification of the encoding operation for one or 
more of encoders 116-120 may be introduced via 
unit 142 in case the video format of one or more of 
video signals 11 -In is changed. Alternatively, unit 
142 may introduce a modified encoding procedure 
into encoders 116-120, for example, when display 
memory 104 is replaced by a smaller display 
memory. Data compression techniques or lower 
resolution encoding may then be introduced via 
unit 142. 

Decoding control unit 148 controls the opera- 
tion of decoding section 106. For instance, unit 148 
may modify the CLUT in the appropriate one of 
decoders 130-134 for changing the pseudo-colour 
-r epresent a t ion. A l t ernati ve l y , un it 148 m a y adapts 



quired to correctly adjust the shades of the colours 
used. 

Memory control unit 146 controls the operation 
of display memory 104. Source information section 
5 124 may be programmed via unit 146. The words 
stored at particular addresses of section 126 and 
section 128 may be retrieved via unit 146 under 
PC 40 control for processing or monitoring in PC 
40. A further application may be to selectively store 

10 words supplied by PC 40 via unit 146 in section 
126 and/or in section 128. 

Still-picture control unit 144 is provided to se- 
lectively prevent writing in address regions of sec- 
tion 126 and/or in section 128 in order to maintain 

75 the stored information. Accordingly, since rfsplay 
memory 104 locally remains unchanged, one or 
more still pictures are created in areas on monitor 
20 that corresponds to the selected address re- 
gions of sections 126 and/or 128. Unit 144 may be 

20 operated upon by PC 40 via bus 140, for instance, 
for storage of information specifying which ones of 
the destination pixels of the composite image in 
monitor 20 are to be kept still. Unit 146 may 
directly control display memory 104 by selectively 

25 preventing memory locations from being overwrit- 
ten. This may be useful when handling a one-time- 
only video signal, i.e., once the video signal has 
supplied a complete image, the signal is termi- 
nated. Alternatively, unit 144 may control encoders 

30 116-120 by selectively putting the encoding proce- 
dure on hold. Encoders 116-120 may be provided 
with a memory function to store the encoded in- 
formation prior to supplying the encoded informa- 
tion to multiplexing means 122. Stopping encoding 

35 then does not produce new pixel information, and 
the same pixels stored in encoders' 116-120 
memories are provided during each new compos- 
ite-image-cycle. Encoders' 116-120 memories then 
are adapted to supply the same pixels more than 

40 once. 

In another embodiment, still-picture control unit 
144 may control the storing of source pixels in 
display memory 104 by selectively assigning a 
"valid" flag or an "invalid" flag to a source pixel 

45 provided by multiplexing means 122. If the current 
source pixel is "valid", display memory accepts the 
source pixel to be stored at a predetermined ad- 
dress. If, on the other hand, the current source 
pixel is said to be "invalid", it is discarded and 

so display memory 104 retains the information already 
stored at the predetermined address. 

PC 40 may provide appropriate clocking to run 
encoding section 102, display memory 104, decod- 
ing section 106 and interface means 108. 

-55. 



the decoding procedure, for example when monitor 
20 is replaced by another one of another type. As 
mentioned above, "gamma-correction" may be re- 
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Second example of system in the invention 

Figures 3 and 4 disclose a second example of 
a system 200 according to the invention. Features 
that correspond to those already discussed under 5 
Figure 1 are denoted by the same reference nu- 
merals. System 200 now specifically combines vid- 
eo signals 11 -In with one another and with com- 
puter-generated graphics data provided at a graph- 
ics bus 210 (feature bus) of, e.g., PC 40, via an 10 
MSMF 12, slightly different from MSMF 10 dis- 
cussed above. 

In Figures 3 and 4, PC 40 has a supervisory 
function as discussed under Figures 1 and 2 and, 
in addition, serves to supply graphics data to be 75 
selectively combined on a pixel-by-pixel basis with 
video signals 11 -In. Functionally, the graphics data 
can be handled similarly to video signals 11 -In. That 
is, the graphics data could be routed to. e.g., input 
114 of encoder 120 in encoding means 102 of 20 
Figure 2, provided that encoder 120 is adapted to 
handle graphics data. This option would not need a 
VGA 70 physically present in PC 40 as the en- 
coder could be provided with a VGA. Alternatively, 
the graphics data could be supplied directly to 25 
multiplexing- means 122 if the graphics data are 
already in the appropriate format, thereby avoiding 
a dedicated encoder. However, graphics data are 
already available in graphics unit 70 on a random 
access basis, as contrasted with a raster scan 30 
basis of typical video signals. Accordingly, graphics 
data are suitable to be introduced selectively and 
directly into display memory 104 via graphics bus 
210, bypassing encoding section 102. 

An appropriate one of decoders 130-134 then 35 
is selected, functionally via format indicating sec- 
tion 128 in a similar way as discussed above, in 
order to convert the graphics data via a CLUT (not 
shown) into a signal for monitor 20. This CLUT 
preferably is programmable, e.g., via decoding 40 
control unit 148. 

Third example 

Figures 5 and 6 illustrate a third example of a 45 
system 300 in the invention. Since graphics data 
are generated on a random-access-basis, the 
memory functionality of graphics unit 70 in Figure 
3 may be physically merged with that of display 
memory 104 in Figure 4. A PC 42 in Figure 5 then 50 
does not need a graphics unit with a VGA and with 
a graphics data memory, since graphics data In a 
predetermined digital format may directly be stored 
in display memory 104. Decoding section 106 then 

-i nc l ud es an appropr i ate d o coder, say, decoder~t34 ss™ 

to transform the graphics information into an output 



Circumventing display memory 104 altogether 
as far as graphics data storage is concerned is 
achieved in an alternative embodiment of system 
300, shown in Figure 6. When graphics data are 
merged with video signals under source- and for- 
mat-control as discussed above, the graphics data 
itself may serve both as a displayable data and as 
a control signal to govern the decoding performed 
on the colour identifiers supplied by display mem- 
ory 104. This is explained as follows. 

The graphics data as produced by PC 42 are 
organized as (N J- 1 )-bit wide words. If the addi- 
tional bit is a E3§ic "1 then the remaining N bits of 
the graphics word are to be interpreted as graphics 
colour information for a particular destination pixel 
and for being processed by decoding section 106. 
If the additional bit is a logic "0\ then all or some 
of the remaining N bits are to be interpreted to 
control the proper decoding of the colour informa- 
tion supplied display memory 104. 

PC 42 produces (N + 1)-bit wide data words in 
a RAM 240, functionally organized as an N-bit wide 
RAM 242 and a single-bit wide RAM 244 whose 
addresses are correlated. When RAM 244 has a 
"1" at a particular address, then the word at the 
corresponding address of RAM 242 denotes a par- 
ticular colour and is to be decodes as. such in 
decoding section 106. When RAM 244 has a "0" at 
a particular address, then the word at the asso- 
ciated address of RAM 242 is to be interpreted as 
a control signal to select the proper decoding pro- 
tocol performed in decoding section 106 for the 
selected data item provided by display memory 
104. 

Decoding section 106 selectively handles both 
graphics data supplied via bus 210 and data items 
supplied by display memory 104. To this end. 
system 300 includes a multiplexer 212 and a mul- 
tiplexer 214. Multiplexer 212 transfers either graph- 
ics data supplied via bus 210 or data hems re- 
ceived from display memory 104 to be decoded by 
decoding section 106. Multiplexer 214 transfers 
either data supplied via bus 210 or data supplied to 
a multiplexer input 216 in order to select an appro- 
priate decoding protocol. Multiplexers 212 and 214 
are controlled via the single bits provided by RAM 
244. When multiplexer 212 is controlled to transfer 
the data received from display memory 104, then 
multiplexer 214 is controlled to transfer the data 
received via bus 210 to a control input of decoding 
section 106, thereby selecting an appropriate one 
of the decoders (not shown) housed in section 106. 
When multiplexer 212 is controlled to transfer the 
data received via bus 210, then multiplexer 214 is 
"controlled lo transfer the signal at its input 216 to 
decoding sect i on 10 6, the r eby se l ecting th e appro - 
priate decoder, e.g., a CLUT (not shown) to inter- 
pret the data provided by multiplexer 212 as a 



signal to be handled by monitor 20 upon DA con- 
version in DAC 138. 



7 



r. <FP ofiniR47Ai i „ 



13 



EP 0 601 647 A1 



14 



colour. The signal at input 216 may be a fixed 
word, or may be alterable by an external agent, or 
may be supplied by PC 42. 

In the embodiment of Figure 6, format indicat- 
ing section 128 of Figure 4 is implemented as RAM 5 
242 in PC 42. Also, source information section 124 
of Figure 4 may be implemented in RAM 240 to 
control a bus 246 as multiplexer 122 of Figure 4, 
and to act in concert with RAM 242 serving as 
format indicating section 128. F gures 1-6 illustrate 10 
some functionalities withir.^e^ -system as separate 
features. However, two or^apre of the functionalit- 
ies may be physically integrated together within a 
single device of the system. 

Note that the term "colour" is used herein to is 
indicate any aspect of an information item that can 
be made visually perceptible as a pixel. For the 
purpose of this invention, colour therefore may 
include aspects such as grey level, shading or any 
other aspect that serves to visually differentiate 20 
image constituents. 

Claims 

1- A system for creating a composite image by 25 
selectively combining a plurality of simulta- 
neously available video signals with one an- 
other on a pixel-by-pixel basis; 
characterized in that the system comprises: 

- input means operative to receive the plu- 30 
rality of video signals, each respective 

one of the video signals having a respec- 
tive video format; 

- encoding means connected to the input 
means for encoding respective ones of 35 
the video signals into data items pro- 
vided at an encoder output, wherein each 
particular one of the data items corre- 
sponds to a particular source pixel and 
functionally has a colour identifier to 40 
identify a colour of the particular source 

~. pixel and a format identifier to identify 
the respective video format; 

- multiplexing means having a multiplexing 
input coupled to the encoder output for 45 
transferring a selected one of the data 
items to a multiplexing output; 

- memory means for storage of the se- 
lected data item, having a memory input 
coupled to the multiplexing output and 50 
having a memory output; 

- decoding means coupled to the memory 
output to receive the selected data item 
and to decode the colour identifier of the 

s elect ed data item for d etermining a col - 55— 

our of a specific one of destination pixels 

of the composite image under control of 
the format identifier of the selected data 



item. 

2. The system of Claim 1, characterized in that 
the memory means comprises: 

- a colour identifier section to store the 
colour identifier of the selected data item; 

- a format identifier section to store the 
format identifier of the selected data 
item. 

3. The system of Claim 2, characterized in that 
the memory means has a source identifier 
section to store a source identifier for control 
of the multiplexing means. 

4. The system of Claim 1, operative to selectively 
combine the video signals with computer-gen- 
erated graphics data on a pixel-by-pixel basis, 
characterized in that the system comprises: 

- control means for specifying whether the 
plurality of video signals or the com- 
puter-generated graphics data is to de- 
termine the colour of the specific des- 
tination pixel; 

- further multiplexing means coupled be- 
tween the memory means and the de- 
coding means for determining the colour 
of the specific destination pixel under 
control of the control means by means 
of: 

- either transferring the colour identifier 
supplied by the memory means to the 
decoding means and transferring the 
graphics data to the decoding means 
for decoding the colour identifier un- 
der control of the graphics data, func- 
tioning as the format identifier; 

- or transferring the graphics data to the 
decoding means for being decoded. 

5. The system of Claim 4, characterized in that 

- the system comprises a graphics data 
generator; and 

- the memory means comprises: 

- a colour identifier section to store the 
colour identifier of the selected data 
item; 

- a format identifier section to store the 
format identifier of the selected data 
item; 

and wherein the format identifier section 
is a functional part of the graphics gener- 
ator. 

6 - — TOe^stenru ot Claire ^characterized in tha^ 
the control means is a second functional part 
of the graphics generator. 
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7. The system of Claim 5 or 6, characterized in 
that the memory means has a source identifier 
section to store a source identifier for control 
of the multiplexing means, the source identifier 
section being a functional part of the graphics 5 
generator. 
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